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WHAT IS CLAIMED IS: 

1. A depth information measurement apparatus for 
measuring depth information at a capture position of a 
5 reference image on the basis of the reference image, 
and at least one peripheral image that forms a stereo 
image pair with the reference image, comprising: 

a plurality of depth information measurement 
means for measuring depth information from the 
J 10 reference image and peripheral image, said plurality of 

: t 

^ depth information measurement means using different 

y measurement methods; 

iri selection means for selecting one of a plurality 

p of depth information for each predetermined region of 

C3 

15 the reference image; and 

m 

composition means for compositing the depth 

r ~j 

information selected by said selection means and 
outputting the composited information as depth 
information at the capture position of the reference 
20 image. 

2- The apparatus according to claim 1, wherein said 
plurality of depth information measurement means having 
different measurement precisions and different 
25 measuring process speeds from each other, for measuring 
a plurality of depth information. 
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3. The apparatus according to claim 1, wherein said 
plurality of depth information measurement means, 
comprising : 

first depth information measurement means for 
measuring first depth information from the reference 
image and the peripheral images- 
second depth information measurement means for 
measuring second depth information from the reference 
image and the peripheral image by a method different 
from said first depth information measurement means. 



4- The apparatus according to claim 3, wherein said 
first depth information measurement means has a higher 
processing speed than said second depth information 
measurement means, and the second depth information has 
higher precision than the first depth information. 



5. The apparatus according to any one of claims 1 to 
4, wherein said selection means selects in accordance 
with a predetermined condition. 



6. The apparatus according to claim 5, wherein the 
predetermined condition is a condition obtained from 
the reference image and/or the peripheral image. 



7. The apparatus according to claim 6, wherein sai 
selection means comprises motion detection means for 
detecting a motion in each predetermined region from 
the reference image, and 

said predetermined condition is based on an 
amount and/or a direction of the motion detected by 
said motion detection means. 

8. The apparatus according to claim 6, wherein one 
of said plurality of depth information measurement 
means comprises reliability detection means for 
detecting a reliability of the measured depth 
information, and 

said predetermined condition is based on the 
reliability detected said reliability detection means 

9. The apparatus according to claim 4, wherein sai( 
selection means comprises motion detection means for 
detecting a motion in each predetermined region from 
the reference image, and 

said selection means selects the first depth 
information for the predetermined region in which the 
detected motion has not less than a predetermined 
amount, and selects the second depth information for 
other regions . ' 



10. The apparatus according to claim 4, wherein said 
first depth information measurement means comprises 
reliability detection means for detecting a reliability 
of the first depth information, and 

said selection means selects the first depth 
information for the predetermined region in which the 
detected reliability has not less than a predetermined 
amount, and selects the second depth information for 
other regions . 

11. The apparatus according to claim 4, wherein said 
second depth information measurement means measures the 
second depth information using a condition which is not 
considered when said first depth information 
measurement means measures the first depth information. 

12. The apparatus according to claim 11, wherein the 
conditions include occlusion by an object. 

13. The apparatus according to any one of claims 1 to 
12, wherein said depth information output from said 
composition means are one of disparity information, a 
disparity map, distance information, and a range image. 



14. A mixed reality presentation system including a 
depth information measurement apparatus cited in any 
one of claims 1 to 13. 

15. A storage medium which stores a program that can 
be executed by a computer apparatus, and makes the 
computer apparatus that has executed the program 
function as a depth information measurement apparatus 
cited in any one of claims 1 to 13. 

16. A depth information measurement method for 
measuring depth information at a capture position of a 
reference image on the basis of the reference image, 
and at least one peripheral image that forms a stereo 
image pair with the reference image, comprising steps 
of: 

a measurement step of measurement a plurality of 
depth information from the reference image and 
peripheral image, using different measurement methods; 

a selection step of selecting one of the 
plurality of depth information for each predetermined 
region of the reference image; and 

a composition step of compositing the depth 
information selected by said selection step and 
outputting the composited information as depth 



information at the capture position of the reference 
image . 

17. The method according to claim 16, wherein said 
measurement step measures the plurality of depth 
information at different measurement precisions and 
different measuring process speeds from each other. 



18, The method according to claim 16, wherein said 
measurement step, comprising: 

first depth information measurement step of 
measuring first depth information from the reference 
image and the peripheral image; 

second depth information measurement step of 
measuring second depth information from the reference 
image and the peripheral image by a method different 
from said first depth information measurement step. 



19. The method according to claim 18, wherein said 
first depth information measurement step utilizes a 
higher processing speed than that of said second depth 
information measurement step, and the second depth 
information has higher precision than that of the first 
depth information . 



20. The method according to any one of claims 1 to 4, 
wherein said selection step selects in accordance with 
a predetermined condition. 



5 21. The method according to claim 20, wherein the 
predetermined condition is a condition obtained from 
the reference image and/or the peripheral image. 

C3 22. The method according to claim 21, wherein said 

Cri 10 selection step comprises motion detection step of 

fg detecting a motion in each predetermined region from 

I y 

the reference image, and 
I ' said predetermined condition is based on an 

amount and/or a direction of the motion detected by 

h '-ST 

15 said motion detection step. 

r "1 

23. The method according to claim 21, wherein said 
measurement step has reliability detection step of 
detecting a reliability of the measured depth 

20 information, and 

said predetermined condition is based on the 
reliability detected said reliability detection step. 

24. The method according to claim 19, wherein said 
25 selection step comprises motion detection step of 
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detecting a motion in each predetermined region from 
the reference image, and 

said selection step selects the first depth 
information for the predetermined region in which the 
5 detected motion has not less than a predetermined 

amount, and selects the second depth information for 
other regions. 

C3 25. The method according to claim 19, wherein said 

Cri 10 first depth information measurement step comprises 

fS S3 

Ca reliability detection step of detecting a reliability 

■Q of the first depth information, and 

-J said selection step selects the first depth 

O 

,^r= information for the predetermined region in which the 

15 detected reliability has not less than a predetermined 

':z amount, and selects the second depth information for 

11 

other regions . 

26. The method according to claim 19, wherein said 
20 second depth information measurement step measures the 
second depth information using a condition which is not 
considered when said first depth information 
measurement step measures the first depth information. 

25 27. The method according to claim 26, wherein the 
conditions include occlusion by an object. 
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28. The method according to any one of claims 1 to 12 
wherein said depth information output from said 
composition step are one of disparity information, a 
disparity map, distance information, and a range image. 

29. A mixed reality presentation system executing a 
depth information measurement method cited in any one 
of claims 16 to 28. 

30. A storage medium which stores a program that can 
be executed by a computer apparatus, and makes the 
computer method that has executed the program function 
as a depth information measurement method cited in any 
one of claims 16 to 28. 

31. An image generation apparatus comprising: 

a first image processor for generating a stereo 
image from a plurality of images having disparity; 

a second image processor capable of generating an 
image with a higher precision than that of said first 
image processor; 

a motion detector for detecting a motion of an 
image; and 

an image generator for generating an image by 
selectively using outputs from said first and second 




image processors on the basis of an output from said 
motion detector. 

32. The apparatus according to claim 31, wherein said 
5 first image processor can generate an image at a higher 

speed than that of said second image processor. 

33. The apparatus according to claim 31, wherein said 
image generator generates an image for a large moving 

10 amount in the image using the output from said first 
image processor and for a small moving amount in the 
image using the output from said second image processor. 

34. The apparatus according to claim 31, wherein said 
15 first and second image processors output depth 

information of an image. 

35. The apparatus according to claim 31, wherein said 
image generator controls composition processing for the 

20 outputs from said first and second image processors on 
the basis of a predetermined output from said first 
image processor. 

36. The apparatus according to claim 35, wherein the 
25 predetermined output comprises information about a 

reliability. 
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37- An image generating method comprising steps of: 

a first image processing step of generating a 
stereo image from a plurality of images having 
disparity; 

5 a second image processing step of generating an 

image with a higher precision than that of said first 
image processing step; 

a motion detecting step of detecting a motion of 
P an image ; and 

ITi 10 an image generating step of generating an image 

JO by selectively using outputs from said first and second 

f y 

image processors on the basis of an output from said 

. ... 

y motion detecting step. 

l,^ 15 38. The method according to claim 31, wherein said 

first image processing step can generate an image at a 

r "*( 

higher speed than that of said second image processing 
step . 

20 39. The method according to claim 37, wherein said 
image generating step generates an image for a large 
moving amount in the image using the output from said 
first image processing step and for a small moving 
amount in the image using the output from said second 

25 image processing step. 
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40. The method according to claim 37, wherein said 
first and second image processing steps output depth 
information of an image, 

41. The method according to claim 37, wherein said 
image generating step controls composition processing 
for the outputs from said first and second image 
processing steps on the basis of a predetermined output 
from said first image processing step. 

42. The method according to claim 41, wherein the 
predetermined output comprises information about a 
reliability. 

43. A computer-readable storage medium which stores a 
program for generating an image, said program 
comprising : 

a first image processing step of generating a 
stereo image from a plurality of images having 
disparity; 

a second image processing step of generating an 
image with a higher precision than that of said first 
image processing step; 

a motion detecting step of detecting a motion of 
an image; and 



m 10 



11 



an image generating step of generating an image 
by selectively using outputs from said first and second 
image processors on the basis of an output from said 
motion detecting step. 

44. The medium according to claim 43, wherein said 
first image processing step can generate an image at a 
higher speed than that of said second image processing 
step. 



45. The medium according to claim 43, wherein said 
image generating step generates an image for a large 
moving amount in the image using the output from said 
first image processing step and for a small moving 

15 amount in the image using the output from said second 
image processing step. 

46. The medium according to claim 43, wherein said 
first and second image processing steps output depth 

20 information of an image. 



47. The medium according to claim 43, wherein said 
image generating step controls composition processing 
for the outputs from said first and second image 
processing step's on the basis of a predetermined output 
from said first image processing step. 
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48. The medium according to claim 47, wherein the 
predetermined output comprises information about a 
reliability. 
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